Rates of Convergence in the Source Coding Theorem , in Empirical Quantizer Design , and in Universal Lossy

Rate of convergence results are established for vector quantization. Convergence rates are given for a n increasing vector dimension a n d / o r a n increasing training set size. I n particular, the following results are shown for memoryless realvalued sources with bounded support a t transmission rate R: (1) If a vector quantizer with fixed dimension k is designed to minimize the empirical mean-square error (MSE) with respect to m training vectors, then its MSE for the true source converges in expectation and almost surely to the minimum possible MSE as O(J-1; (2) The MSE of a n optimal k-dimensional vector quantizer for the true source converges, a s the dimension grows, to the distortion-rate function D ( R ) as O ( d m ) ; (3) There exists a fixed-rate universal lossy source coding scheme whose per-letter MSE on n real-valued source samples converges in expectation and almost surely to the distortion-rate function D ( R ) as O(\lloglog n / log A 1; (4) Consider a training set of n real-valued source samples blocked into vectors of dimension k, and a k-dimension vector quantizer designed to minimize the empirical MSE with respect to the m = In / k J training vectors. Then the per-letter MSE of this quantizer for the true source converges in expectation and almost surely to the distortion-rate function D(R) a s O(4loglogn / logn 1, if one chooses k = 1(1 / R ) ( 1 ~ ) l o g n ] for any E E (0,I).